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Abstract 


The  Chemistry  Division  of  the  Naval  Research  Laboratory  has  developed  the  Multi-sensor 
Towed  Array  Detection  System,  MTADS,  to  address  the  problem  of  buried  UXO  detection  and 
characterization  with  support  from  the  Environmental  Security  Technology  Certification 
Program,  ESTCP.  The  original  MTADS  incorporates  both  cesium  vapor  full-field 
magnetometers  and  pulsed-induction  sensors  in  linear  arrays  that  are  towed  over  survey  sites  by 
an  all-terrain  vehicle.  More  recently,  man-portable  and  airborne  versions  of  the  system  have 
been  demonstrated.  Sensor  positioning  is  provided  by  state-of-the-art  Real  Time  Kinematic 
(RTK)  GPS  receivers.  The  survey  data  acquired  by  MTADS  are  preprocessed  using  tools  from 
the  Geosoft  Oasis  montaj  suite  and  then  targets  are  analyzed  using  an  NRL-developed  Data 
Analysis  System,  DAS.  The  perfonnance  of  the  MTADS  has  been  demonstrated  at  a  number  of 
prepared  sites  and  live  ranges  over  the  past  five  years.  The  vehicular  and  man-portable  versions 
can  detect  and  locate  ordnance  with  accuracies  on  the  order  of  15  cm  while  the  airborne  version 
has  demonstrated  location  accuracy  of  25  -  30  cm. 

In  February  2003,  we  demonstrated  the  vehicular  and  airborne  magnetometer  systems  on 
approximately  1550  acres  at  Target  SI  on  the  Pueblo  of  Isleta,  near  Albuquerque  NM.  This 
Demonstration  had  three  primary  objectives.  First,  we  demonstrated  the  airborne  MTADS  in  a 
new  geological  environment  against  a  new  target  set.  Second,  the  probability  of  detection  and 
location  accuracy  of  the  airborne  system  was  measured  against  the  baseline  vehicular  system. 

The  targets  for  this  comparison  included  the  existing  practice  bombs  as  well  as  an  emplaced 
target  suite  chosen  by  ESTCP.  And,  finally,  the  MTADS  airborne  system  was  measured  against 
the  system  developed  and  deployed  by  Oak  Ridge  National  Lab.  This  intersystem  comparison 
will  allow  the  Institute  for  Defense  Analyses  to  quantify  the  performance  features  of  each 
system. 

The  Demonstration  consisted  of  three  overlapping  surveys.  First,  a  vehicular  magnetometery 
survey  of  100  acres  near  the  previously-identified  bull’s  eye,  SI.  Targets  on  this  area  were 
expected  to  be  M38  and  BDU-33  practice  bombs  and  an  array  of  ordnance  to  be  emplaced  by  the 
Army  Engineering  R&D  Center  at  the  direction  of  the  ESTCP  Program  Office.  Second,  an 
airborne  magnetometry  survey  of  1500  acres  around  the  target  to  include  the  vehicular  area. 
Finally,  an  airborne  survey  by  Oak  Ridge  National  Lab  of  the  same  1500  acres. 

The  vehicular  MTADS  system  covered  28.1  hectares  (69.5  acres)  including  a  10-m  buffer 
around  the  vehicular  site.  In  this  area,  we  marked  1364  targets  (16  of  these  were  the  calibration 
targets  planted  in  the  upper  part  of  the  site)  for  remediation  using  the  now  familiar  classification 
scheme  of  1  for  high-confidence  ordnance,  2  for  medium-confidence  ordnance,  3  for  low- 
confidence  ordnance,  4  for  low-confidence  clutter,  5  for  medium-confidence  clutter,  and  6  for 
high-confidence  clutter.  The  airborne  MTADS  system  surveyed  570  hectares  (1408  acres). 
Airborne  targets  were  picked  in  two  areas.  The  first  area  was  the  100-acre  vehicular  site 
excluding  the  densest  target  area  surrounding  the  bull’s  eye  which  resulted  in  1260  targets.  The 
second  area  in  which  targets  were  picked  was  designated  the  “Primary  Area.”  This  area  was 
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chosen  by  the  ESTCP  Program  Office  in  conjunction  with  the  two  survey  teams.  Target  analysis 
in  this  area  resulted  in  388  targets. 

Working  from  the  target  lists  submitted  by  each  of  the  three  survey  teams  (MTADS  vehicular, 
MTADS  airborne,  and  ORNL  airborne)  analysts  from  the  Institute  for  Defense  Analyses,  IDA, 
selected  targets  for  remediation  in  the  two  airborne  areas  mentioned  above,  the  Vehicular  Area 
and  the  Primary  Area.  Six  hundred  ninety  (711  original  picks  -  2 1  inadvertently  included  seed 
targets)  targets  were  selected  for  remediation  by  UXO  technicians  from  EOTI,  Inc.,  our 
remediation  contractor.  Each  of  the  remediated  items  was  characterized,  photographed,  and 
located  by  GPS  re-survey.  A  number  of  the  targets  in  the  “Primary  Area”  were  reported  as  “no 
find”  by  the  EOD  teams.  Fifty  of  these  targets  were  re-examined  in  a  follow-up  survey  by 
members  of  the  vehicular  team  in  January,  2004. 

Using  the  results  of  this  remediation,  we  develop  probability  of  detection  statistics  for  each  of  the 
two  MTADS  systems,  derive  information  about  location  accuracy,  and  examine  the  classification 
performance  of  the  two  systems. 
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MTADS  Airborne  and  Vehicular  Survey  of 
Target  SI  at  Isleta  Pueblo 

Albuquerque,  NM 
17  February  -  2  March,  2003 


1.  Introduction 


1.1  Background 

Buried  unexploded  ordnance,  UXO,  is  one  of  the  Department  of  Defense’s  most  pressing 
environmental  problems.  Not  limited  to  active  ranges  and  bases,  UXO  contamination  is  present 
at  DOD  sites  that  are  dormant  and  in  areas  adjacent  to  military  ranges  that  are  under  the  control 
of  other  government  agencies  and  the  private  sector. 

Traditional  methods  for  buried  UXO  detection,  characterization,  and  remediation  are  labor- 
intensive,  slow  and  inefficient.  Typical  detection  and  characterization  methods  rely  on  hand¬ 
held  detectors  operated  by  UXO  technicians  who  slowly  walk  across  the  survey  area.  This 
process  has  been  documented  as  inefficient  and  marginally  effective.1  In  addition,  a  large 
portion,  approaching  70%  in  some  cases,  of  the  total  budget  of  a  typical  remediation  effort  is 
spent  on  digging  targets  that  do  not  turn  out  to  be  ordnance. 

The  Environmental  Security  Technology  Certification  Program,  ESTCP,  has  supported  the  Naval 
Research  Laboratory  in  the  development  of  the  Multi-sensor  Towed  Array  Detection  System, 
MTADS,  to  address  these  deficiencies.  The  MTADS  incorporates  both  cesium  vapor  full-field 
magnetometers  and  pulsed-induction  sensors  in  linear  arrays  that  are  towed  over  survey  sites  by 
an  all-terrain  vehicle.  Sensor  positioning  is  provided  by  state-of-the-art  Real  Time  Kinematic 
(RTK)  GPS  receivers.  The  survey  data  acquired  by  MTADS  are  preprocessed  using  tools  from 
the  Geosoft  Oasis  montaj  suite  and  then  targets  are  analyzed  using  an  NRL-developed  Data 
Analysis  System,  DAS.  The  DAS  was  designed  to  locate,  identify  and  categorize  all  military 
ordnance  at  its  maximum  self-burial  depth.  It  is  efficient  and  simple  to  operate  by  relatively 
untrained  personnel.  The  perfonnance  of  the  MTADS  has  been  demonstrated  at  a  number  of 
prepared  sites  and  live  ranges  over  the  past  five  years.""  "  It  can  detect  and  locate  ordnance  with 
accuracies  on  the  order  of  15  cm.5 

Many  sites  of  interest  have  terrain  that  cannot  be  traversed  by  a  vehicle  or  on  foot.  Some  sites, 
particularly  on  active  ranges,  are  cluttered  with  a  variety  of  ordnance  that  make  clearance  or  even 
characterization  activities  potentially  dangerous.  Finally,  there  are  many  fonnerly  used  ranges 
dating  from  World  War  II  (and  earlier)  that  are  located  in  areas  involving  tens  or  hundreds  of 
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thousands  of  acres  with  isolated  bombing  targets  or  impact  ranges.  Locations  of  many  of  these 
impact  areas  (or  ordnance  burial  caches)  are  unknown  or  imprecisely  located.  Some  of  these 
areas  are  located  on  Native  American  reservations  while  others  involve  BRAC  or  pending 
BRAC  sites. 

To  address  these  issues  NRL,  under  ESTCP  Project  200031,  has  developed  and  demonstrated  an 
airborne  adjunct  to  the  MTADS  vehicular  and  man-portable  systems.  This  airborne  system 
allows  efficient  and  effective  characterization  of  many  of  these  areas  that  are  inappropriate  for 
vehicular  surveys.  It  is  designed  to  rapidly,  economically,  and  efficiently  survey  large  sites  that 
are  not  appropriate  for  vehicular  or  man-portable  surveys.  While  the  airborne  system  is  not 
capable  of  detecting  the  smallest  classes  of  buried  UXO  at  their  maximum  likely  self-burial 
depths,  it  does  allow  efficient  surveys  of  large  areas  to  locate  target  bull's  eyes,  impact  clusters, 
and  burial  caches.  Under  favorable  conditions  we  can  detect  and  characterize  individual 
ordnance  such  as  GP  bombs  and  the  projectiles  larger  than  60-inm  mortars.  For  these  individual 
ordnance  targets,  the  system  estimates  burial  depths,  likely  ordnance  size,  and  provides  for  target 
way  pointing,  as  well  as  creating  GIS-compatible  target  output  maps  and  sorted  target  tables. 

The  primary  goals  of  the  airborne  MTADS  Dem/Val  program  are  enumerated  below: 

1 .  Field  an  airborne  magnetometer  array  for  efficiently  surveying  very  large  or  inaccessible 
areas, 

2.  The  system  should  have  the  capability  to  characterize  the  presence  of  UXO  associated 
with  impact  bull's  eyes  or  buried  ordnance  caches, 

3.  The  airborne  survey  system  will  incorporate  many  of  the  successful  developments 
associated  with  the  vehicular  MTADS,  including  sensors,  satellite-based  navigation, 
efficient  data  acquisition  approaches,  and  the  DAS  suite  of  utilities  for  data  manipulation 
and  target  analysis, 

4.  The  system  should  create  a  permanent  record  in  global  coordinates  of  the  positions  of  all 
targets,  and 

5 .  The  intended  use  of  this  airborne  automated  technology  is  for  site  characterization  of 
DoD  bombing  and  target  ranges.  The  system  must  be  capable  of  efficiently  and  rapidly 
surveying  relatively  large  areas  typical  of  ranges  used  during  and  since  WW  II  that 
occupy  millions  of  acres. 

1.2  Objective  of  the  Demonstration 

This  Demonstration  had  three  primary  objectives.  First,  we  demonstrated  the  airborne  MTADS 
in  a  new  geological  environment  against  a  new  target  set.  Second,  the  probability  of  detection 
and  location  accuracy  of  the  airborne  system  was  measured  against  the  baseline  vehicular 
system.  The  targets  for  this  comparison  included  the  existing  practice  bombs  as  well  as  an 
emplaced  target  suite  chosen  by  ESTCP.  And,  finally,  the  MTADS  airborne  system  was 
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measured  against  the  system  developed  and  deployed  by  Oak  Ridge  National  Lab.  This 
intersystem  comparison  will  allow  the  Institute  for  Defense  Analyses  to  quantify  the 
performance  features  of  each  system. 

1.3  Official  DOD  Requirement  Statement 

The  Navy  Tri-Service  Environmental  Quality  Research  Development  Test  and  Evaluation 
Strategic  Plan  specifically  addresses  under  Thrust  Requirements  l.A.  1  and  1  .A. 2,  the 
requirements  for  improved  detection,  location  and  removal  of  UXO  on  land  and  under  water. 

The  index  numbers  associated  with  these  requirements  are  l.I.4.e  and  l.III.2.f.  The  priority  1 
rankings  of  these  requirements  indicate  that  they  address  existing  statutory  requirements, 
executive  orders  or  significant  health  and  safety  issues.  Specifically  the  requirements  document 
states: 

There  are  more  than  twenty  million  acres  of  bombing  and  target  ranges  under  DOD 
control.  Of  particular  concern  for  the  Navy  are  the  many  underwater  sites  which  have 
yet  to  be  characterized.  Each  year  a  significan  t  fraction  (200,000-500,000  acres)  of 
these  spaces  are  returned  to  civilian  (Private  or  Commercial)  use.  All  these  areas  must 
be  surveyed  for  buried  ordnance  and  other  hazardous  materials,  rendered  certified  and 
safe  for  the  in  tended  end  use.  This  is  an  extremely  labor  in  tensive  and  expensive  process, 
with  costs  often  far  exceeding  the  value  of  the  land....  Improved  technologies  for  locating, 
identifying  and  marking  ordnance  items  must  be  developed  to  address  all  tvpes  of  terrain, 
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such  as  open  fields,  wooded  areas,  rugged  inaccessible  areas,  and  underwater  sites. 

The  MTADS  vehicular  system  addresses  all  aspects  of  the  Tri-Service  Requirements  for  land- 
based  buried  UXO.  It  is  designed  to  survey  large  sites  rapidly  and  efficiently,  with 
commensurate  economic  benefits.  Moreover,  it  is  capable  of  detecting  all  classes  of  buried  UXO 
at  their  likely  self-burial  depths.  The  detection  capability  of  the  airborne  system  is  reduced  for 
smaller  targets  as  will  be  demonstrated  below.  Both  systems  will  correctly  locate  buried  targets, 
determine  their  burial  depths,  classify  the  likely  ordnance  size,  provide  for  future  target  way 
pointing,  as  well  as  create  GIS-compatible  target  output  maps  and  sorted  target  tables. 


2.  Technology  Description 

2.1  Technology  Development  and  Application 
2.1.1  Vehicular  Magnetometer  System 

The  MTADS  hardware  consists  of  a  low-magnetic-signature  vehicle  that  is  used  to  tow  linear 
arrays  of  magnetometer  and  pulsed-induction  sensors  to  conduct  surveys  of  large  areas  to  detect 
buried  UXO.14  The  MTADS  tow  vehicle,  manufactured  by  Chenowth  Racing  Vehicles,  is  a 
custom-built  off-road  vehicle,  specifically  modified  to  have  an  extremely  low  magnetic 
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signature.  Most  ferrous  components  have  been  removed  from  the  body,  drive  train  and  engine 
and  replaced  with  non-ferrous  alloys. 

The  MTADS  magnetometers  are  Cesium-vapor  full-field  magnetometers  (Geometries  Model 
822ROV)  selected  for  low  noise  and  inter-sensor  reproducibility.  An  array  of  eight  sensors  is 
deployed  as  a  magnetometer  array  on  an  aluminum  and  composite  platform,  Figure  1.  The 
sensors  are  sampled  at  50  Hz  and  typical  surveys  conducted  at  6  mph;  this  results  in  a  sampling 
density  of  ~6  cm  along  track  with  a  sensor  spacing  of  25  cm.  The  time-variation  of  the  Earth’s 
field  is  measured  by  a  ninth  sensor  deployed  at  a  static  site  removed  from  the  survey  area.  These 
data  are  used  to  correct  the  survey  magnetic  readings. 


Fig.  1-  Vehicular  MTADS  magnetometer  system  deployed  at 
Isleta  Pueblo,  Albuquerque,  NM 


The  sensor  positions  are  measured  in  real-time  (5  Hz)  using  Real  Time  Kinematic  (RTK)  Global 
Positioning  System  (GPS)  technology  which  results  in  position  accuracies  of  ~5  cm.  All 
navigation  and  sensor  data  are  time-stamped  with  Universal  Coordinated  Time  (UTC)  derived 
from  the  satellite  clocks  and  recorded  by  the  data  acquisition  computer  (DAQ)  in  the  tow 
vehicle.  The  sensor,  position,  and  timing  files  are  downloaded  periodically  throughout  a  survey 
onto  magnetic  disks  and  transferred  to  the  Data  Analysis  System  (DAS)  for  analysis. 

2.1.2  Airborne  Magnetometer  System 

The  airborne  MTADS  system  hardware  includes  an  array  of  seven  total  field  magnetometers 
mounted  on  a  Bell  Helicopter  Model  206L  series  “Longranger.”  The  MTADS  magnetic  sensors 
are  Cs  vapor  full-field  magnetometers  (a  variant  of  the  Geometries  822  sensor,  designated  as  the 
Model  822A).  The  helicopter  with  the  magnetometer  array  mounted  is  shown  in  Figure  2.  All 
sensors  are  interfaced  to  a  data  acquisition  computer  (DAQ).  The  DAQ  electronics  are  contained 
in  a  rack  mounted  in  the  rear  starboard  seat  position  in  the  helicopter,  Figure  3.  The  interface  to 
the  helicopter  power  and  power  distribution  system  is  also  in  the  rack,  as  are  readouts  for  all  the 
sensor  inputs.  The  survey  progress  is  monitored  continually  by  an  operator  in  the  rear  port  seat. 
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Fig.  2  -  MTADS  Airborne  magnetometer  system  deployed  at 
Isleta  Pueblo,  NM 


In  the  9-meter  boom,  the  seven  sensors  are  mounted  with  a  1.5  meter  horizontal  spacing.  The 
specially-selected  magnetometers,  which  are  airborne  quality,  were  acceptance  tested  at  the 
manufacturer’s  facility  to  verify  sensitivity,  sensor  noise,  heading  error,  dead  zones,  inter-sensor 
compatibility,  and  performance  with  the  multi-sensor  interface  modules. 

The  sensor  positions  above  the  surface  of  the  Earth  (latitude,  longitude,  and  height  above 
ellipsoid)  are  determined  using  satellite-based  GPS  navigation,  employing  the  latest  Real  Time 
Kinematic  (RTK)  technology,  which  provides  a  real-time  position  update  (at  20  Hz)  with  an 
accuracy  of  about  5  cm.  GPS  satellite  clock  time  is  used  to  time-stamp  both  position  and  sensor 
data  information  for  later  correlation. 


Fig.  3  -  MTADS  airborne  DAQ  mounted  in 
the  rear  seat  of  the  survey  helicopter 
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The  helicopter  pilot  flies  the  survey  using  an  onboard  navigation  guidance  display,  Figure  4. 

The  survey  parameters  are  set  up  in  a  second  computer  that  supports  the  pilot  display.  This 
computer  shares  the  GPS  navigation  data  with  the  DAQ.  The  survey  guidance  display  provides 
left-right  of  track  indicator,  altitude  indicator,  an  automatic  line  number  increment,  an 
adjustment  for  lateral  offset,  a  color-coded  flight  swath  overlay,  and  the  ability  to  zoom  in  or  out 
on  the  display.  The  survey  course-over-ground  (COG)  is  plotted  for  the  pilot  in  real  time  on  the 
display,  as  are  presentations  showing  the  data  quality  for  the  altimeter  and  GPS  and  the  GPS 
navigation  fix  quality.  This  allows  the  operator  to  respond  to  both  visual  cues  on  the  ground  and 
to  the  survey  guidance  display.  Following  a  survey,  the  operator  can  survey  any  missed  areas 
before  leaving  the  site. 


Fig.  4  -  Two  views  of  the  pilot 
guidance  system 


2.1.3  Data  Analysis  Methodology 

For  the  vast  majority  of  MTADS  surveys,  the  MTADS  Data  Analysis  System  has  been  used  to 
convert  the  sensor  and  position  data  files  into  an  anomaly  map  by  interpolating  the  individual 
sensor  readings  using  the  GPS-derived  positions.  The  DAS  software  was  developed  specifically 
for  the  MTADS  program  as  a  stand-alone  suite  of  programs.  PC-based  code  is  now  available 
and  has  been  used  for  recent  operation.  The  DAS  is  written  for  use  by  both  sophisticated  and 
novice  users.  Even  the  novice  can  perfonn  a  complete  anomaly  analysis  using  menu-driven 
tools  and  default  settings.  For  the  advanced  user,  there  is  an  extensive  range  of  options  available 
including  navigation  data  cleanup,  sensor  nulling  and  leveling,  noise  filtering,  etc.  A  working 
screen  of  the  DAS  is  shown  in  Figure  5. 

In  the  case  of  isolated  ordnance  targets  in  the  far  field  (i.e.  farther  from  the  sensors  than  their 
characteristic  dimension)  the  DAS  employs  resident  physics-based  models  to  determine  target 
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Fig.  5  -  Working  screen  of  the  MTADS  DAS 
showing  the  project  view  on  the  left  and  an  expanded 
analysis  view  on  the  right 


size,  position,  and  depth.  Extensive  data  sets  have  been  acquired  and  processed  to  calibrate  these 
models.  Using  these  models,  we  have  demonstrated  probabilities  of  detection  of  95  to  97%  and 
location  accuracies  of  15  cm  with  the  vehicular  magnetometer  system. 

Although  we  have  achieved  impressive  results  using  the  DAS,  it  has  proven  difficult  to  transition 
to  the  general  UXO  user  community.  Beginning  with  the  demonstrations  of  the  airborne  system, 
we  have  performed  the  data  preprocessing  functions  through  generation  of  mapped  data  files 
using  a  commercial  software  package,  Geosoft  Oasis  montaj™.  An  example  of  a  working  screen 
from  montaj™  is  shown  in  Figure  6.  The  upper  panel  of  the  screen  shows  a  portion  of  the  Oasis 
database,  the  middle  shows  corrected  and  uncorrected  plots  of  one  of  the  sensor  tracks  and  the 
lower  panel  shows  a  detail  of  the  interpolated  sensor  data. 


Fig.  6  -  A  working  screen  of  Oasis  montaj™  showing  airborne 
data  from  this  Demonstration 
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At  this  point  in  our  development,  we  import  the  mapped  data  files  generated  using  montaj™  into 
the  DAS  for  target  selection  and  analysis.  We  are  in  the  process  of  converting  the  analysis 
routines  developed  under  ESTCP  and  SERDP  sponsorship  to  Geosoft  GXs,  executable  files  that 
can  be  called  from  the  Oasis  environment.  This  will  allow  us  to  perform  the  entire  data  analysis 
from  input  of  raw  data  files  through  data  quality  checks,  mapping  of  individual  sensor  readings, 
target  selection,  model  fit,  and  finally  generation  of  target  lists  and  output  graphics  in  the  Oasis 
environment.  All  target  analyses  reported  here  were  accomplished  using  routines  in  the  MTADS 
DAS. 

2.2  Previous  Testing  of  the  Technology 

As  mentioned  above,  the  vehicular  MTADS,  which  will  serve  as  the  comparison  benchmark  for 
this  Demonstration,  has  been  tested  at  a  wide  variety  of  sites  over  the  past  six  years."'  Of 
particular  relevance  is  the  demonstrated  ability  of  the  vehicular  system  to  locate  ordnance  of  the 
type  to  be  encountered  here  with  a  mean  location  error  of  12  cm  and  a  95%  location  distance  of 
29  cm.5  Thus,  the  vehicular  MTADS  magnetometer  system  is  ideal  as  a  perfonnance 
benchmark. 

The  airborne  MTADS  adjunct  has  been  demonstrated  three  times  at  two  sites  over  the  past  year 
and  a  half.  The  first  full-scale  demonstration  was  at  the  Badlands  Bombing  Range  on  the  Pine 
Ridge  Reservation  in  South  Dakota  in  September  2001.  During  this  demonstration  a  10-acre  site 
seeded  with  25  inert  projectiles  (105-mm,  155-mm,  and  8-inch)  was  flown  to  allow  comparison 
of  the  system  performance  with  that  of  the  vehicular  MTADS,  which  surveyed  the  same  site.  An 
additional  1600  acres  were  surveyed  using  the  airborne  system  as  part  of  continued  clean-up 
efforts  over  the  entire  Impact  Area.  Analysis  of  the  airborne  data  collected  over  the  seeded  site 
resulted  in  a  total  of  161  targets  selected  for  digging  including  all  of  the  seeded  projectiles  and 
one  live,  HE  filled,  155-mm  projectile.15  The  false  alarm  ratio  for  this  site  was  161/26  =  6.2.  A 
total  of  1,193  targets  were  analyzed  from  the  1600  acre  survey,  resulting  in  528  excavations  and 
recovery  of  a  total  of  19  live  UXO  projectiles  including  eleven  155-mm  and  eight  8-inch 
projectiles.15 

The  second  Demonstration  of  the  airborne  system  was  at  Aberdeen  Proving  Ground  in  late  July 
2002.  This  demonstration  involved  an  airborne  survey  of  a  total  of  550  acres  over  selected  sites, 
including  a  94-acre  calibration  area  and  770  additional  acres  over  areas  with  varying  terrain 
types,  and  UXO/clutter  contamination  levels.  At  this  time,  results  are  only  available  from  the 
Phillips  Air  Field  area,  which  was  a  seeded  area  containing  105-mm  projectiles  and  60-  and  81- 
mm  mortars.  Even  though  the  mortars  are  near,  or  below,  the  reliable  detection  threshold  for  the 
airborne  system  depending  on  geologic  and  system  noise  levels,  the  airborne  MTADS  achieved 
an  overall  Pd  of  0.85  which  comprised  1.0  for  the  105-mm  projectiles,  and  0.67  for  each  of  the 
mortar  sizes.  This  level  of  detection  was  achieved  because  the  site  conditions  allowed  for 
extremely  low  survey  altitudes  (sensors  at  lm  or  less).  Further  reporting  awaits  the  results  of  a 
series  of  excavations  in  progress  on  targets  reported  by  the  NRL  and  Oak  Ridge  systems. 
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The  most  recent  Demonstration  of  the  airborne  MTADS  was  again  at  the  Badlands  Bombing 
Range  in  September  2002.  This  demonstration  was  a  combined  vehicular,  man-portable,  and 
airborne  survey  of  the  remaining  area  of  the  Impact  Area  not  covered  in  the  2001  survey.  The 
survey  data  collected  in  this  demonstration  have  been  analyzed  and  reported.16  Remediation 
funding  is  not  available  at  this  time  for  the  targets  analyzed  in  the  Demonstration. 

2.3  Factors  Influencing  Cost  and  Performance 

The  largest  single  factor  affecting  the  airborne  MTADS  survey  costs  and  production  rates  is  the 
cost  of  operating  the  survey  helicopter  on  site.  During  recent  surveys,  we  have  paid 
approximately  $700  per  hour  with  a  four  hour  daily  minimum.  Mobilization  of  the  aircraft  to 
and  from  the  site  from  its  home  base  is  charged  at  these  same  rates.  Therefore,  to  maximize 
production  at  minimum  cost,  surveys  must  be  arranged  with  long  survey  lines  to  minimize  the 
time  spent  in  turns,  frequent  examinations  of  data  quality  to  minimize  time  spent  taking 
unacceptable  data,  minimal  time  lost  to  aircraft  refueling  by  having  fuel  available  on  site,  and 
aircraft  basing  to  minimize  daily  ferry  to  and  from  the  survey  site. 

2.4  Advantages  and  Limitations  of  the  Technology 

Unlike  the  vehicular  magnetometer  system,  the  airborne  system  is  not  capable  of  detecting  the 
smallest  classes  of  buried  UXO  at  depth.  However,  we  have  taken  data  from  Target  N9  on  the 
Laguna  Pueblo,  which  has  over  15,000  M3 8  practice  bombs,  resampled  the  data  and  modeled  the 
magnetic  response  that  would  be  observed  by  an  airborne  sensor  array  above  the  ground.  While 
the  magnetic  signals  are  spatially  spread  and  diminished  in  intensity  as  the  sensors  move  further 
above  the  ground,  our  results  indicate  that,  at  an  altitude  of  2  meters  above  the  ground,  the 
system  should  be  capable  of  detecting  BDU-33s  or  Mk  82s  in  all  but  the  most  active  geological 
environments  and  ordnance  targets  equivalent  to  or  larger  than  2. 75 -in  warheads  in  geologically 
quiet  areas. 

In  practice,  the  absolute  limit  of  detection  is  limited  by  the  background  noise  level,  which  is  a 
combination  of  the  geophysical  noise  and  the  platform-induced  noise  from  the  helicopter.  The 
treatment  of  magnetometry  data  to  correct  for  airborne  platform-induced  signals  uses  a  standard 
technique  called  aeromagnetic  compensation.  This  technique  uses  commercially-available 
equipment  and  reduces  the  platform-induced  magnetic  noise  for  fixed  wing  aircraft  to  on  the 
order  of  0.01  nT.  This  approach  has  been  widely  used  in  the  geophysical  exploration  community 
on  both  fixed  wing  aircraft  and  for  helicopters.  Depending  on  the  techniques  used,  and  the  type 
of  platform,  the  compensation  can  reduce  the  platform  and  heading  noise  to  0. 1-0.5  nT.  This  is 
well  below  the  typical  geophysical  noise  due  to  magnetic  soils  and  rocks.  The  signal  intensity 
from  an  individual  ordnance  item  the  size  of  a  GP  bomb  (or  an  individual  cache  of  ordnance)  is  a 
few  tens  to  several  hundred  nT,  even  at  several  meters  altitude.  The  ability  to  detect  and 
characterize  an  isolated  large  target  is  therefore  not  a  matter  of  signal  strength  or  signal-to-noise 
ratio,  but  a  matter  of  having  a  data  sampling  density  high  enough  to  identify  the  target  as  a  target 
and  to  characterize  its  magnetic  anomaly  signature  using  dipole-fitting  routines.  These  issues 
were  incorporated  into  the  design  of  the  horizontal  array  spacing  and  the  flying  speed. 
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On  large  open  ranges  the  vehicular  MTADS  provides  an  efficient  survey  technology.  Surveys 
with  the  magnetometer  array  often  exceed  production  rates  of  20  acres  per  day.  When  a  site  has 
vegetation  cover  or  topography  that  precludes  vehicular  traffic,  the  man-portable  adjunct 
MTADS  can  often  be  used.  However,  there  are  sites  that  cannot  be  traversed  on  foot,  others  that 
are  dangerous,  and  still  others  that  contain  isolated  bombing  targets  or  impact  ranges,  located  at 
best  imprecisely,  within  tens  or  hundreds  of  thousands  of  acres.  For  these  sites,  the  Airborne 
MTADS  will  produce  much  more  rapid  and  efficient  surveying,  with  the  commensurate 
economic  benefits.  The  Airborne  MTADS  is  capable  of  survey  production  rates  of  50 
acres/hour. 

The  helicopter  is  typically  flown  at  a  low  altitude  (1.5-5  meters),  with  an  array  horizontal 
sensor  spacing  of  1.5  meters,  and  the  forward  velocity  of  10  -  20  meters  per  second.  To  achieve 
this,  the  sensors  have  been  fixed  to  hard  points  on  the  helicopter.  As  seen  in  Figure  2,  the  sensor 
boom  extends  well  in  front  of  and  is  clearly  visible  to  the  pilot;  this  is  important  for  low  altitude 
flights.  With  the  sensor  spacing  of  1.5  meters,  a  data  collection  rate  of  100  Hz,  and  a  speed  over 
ground  of  20  m/sec,  the  data  density  provides  50  data  points  on  a  typical  target  to  fit  the  dipole 
signature.  Any  yaw  in  the  helicopter  attitude  during  surveying  decreases  the  effective  sensor 
spacing,  requiring  adjacent  survey  lines  to  be  flown  closer  together. 
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3.  Demonstration  Design 


3.1  Performance  Objectives 

This  Demonstration  consisted  of  three  overlapping  surveys.  First,  a  vehicular  magnetometery 
survey  of  100  acres  near  the  previously-identified  bull’s  eye,  SI,  on  the  Isleta  Pueblo  near 
Albuquerque,  NM  was  planned.  Targets  on  this  area  were  expected  to  be  M38  and  BDU-33 
practice  bombs  and  an  array  of  ordnance  to  be  emplaced  by  the  Army  Engineering  R&D  Center 
at  the  direction  of  the  ESTCP  Program  Office.  Second,  an  airborne  magnetometry  survey  of 
1500  acres  around  the  target  to  include  the  vehicular  area.  Finally,  an  airborne  survey  by  Oak 
Ridge  National  Lab  of  the  same  1500  acres. 

The  vehicular  results  will  be  used  as  a  comparison  benchmark  for  the  results  of  the  two  airborne 
systems.  Consequently,  all  the  targets  within  the  vehicular  survey  were  to  be  analyzed  and  fit. 

In  practice,  only  ~64  acres  were  completely  surveyed  by  the  vehicular  system  as  will  be 
discussed  below.  Target  analysis  of  these  survey  data  identified  1364  targets  that  were  analyzed 
and  reported  to  ESTCP  and  IDA.  Each  of  the  airborne  survey  teams  independently  analyzed 
their  data  using  the  same  target-ranking  scheme  (i.e.  likelihood  of  UXO  vs.  non-UXO  on  a  scale 
of  1  to  6)  as  the  vehicular  system.  All  three  analyses  were  submitted  to  ESTCP  and  IDA  at  the 
conclusion  of  the  surveys  as  an  Excel  (*.xls)  file. 

From  these  analyses,  an  inclusive  dig  list  was  prepared  by  IDA  as  an  Excel  file  and  submitted  to 
NRL  for  transmission  to  the  dig  teams.  Each  target  was  to  be  excavated,  precisely  located  using 
GPS,  documented,  and  photographed.  OE  scrap  was  collected  for  later  certification  and 
disposal.  Recovered  live  ordnance  was  handled  at  the  discretion  of  the  UXO  supervisor  on  site, 
primarily  by  blowing  in  place.  All  excavations  were  filled,  tamped,  and  returned  to  grade. 

The  specific  objective  of  this  Demonstration  was  to  produce  a  quantitative  comparison  among 
the  airborne  systems,  including  probability  of  detection  and  false  alarm  rate.  These  quantities 
will  be  calculated  as  a  function  of  threshold  parameter  where  possible  so  that  an  ROC-type 
analysis  can  be  performed. 

3.2  Selecting  the  Test  Site 

The  location  for  this  Demonstration  was  chosen  by  the  ESTCP  Project  Office  in  conjunction 
with  the  Environment  Department  of  the  Pueblo  of  Isleta.  Subsequent  to  the  choice  of  sites, 
representatives  of  the  NRL  and  Oak  Ridge  teams  met  at  the  site  with  Mr.  Jim  Piatt,  the  Director 
of  the  Environment  Department,  walked  several  of  the  possible  targets,  and  settled  on  Target  SI 
for  this  survey.  This  target  was  chosen  because  it  is  of  most  concern  to  the  tribe,  it  has  the 
greatest  possibility  of  containing  HE-filled  UXO  since  the  tribe  has  located  some  heavy-wall 
fragments  on  the  site,  and  it  offers  the  opportunity  to  survey  the  largest  area  within  the  available 
resources. 
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3.3  Test  Site  History/Characteristics 

“The  Pueblo  of  Isleta  is  located  in  north-central  New  Mexico,  approximately  10  miles  south  of 
Albuquerque.  The  Reservation  is  bordered  on  the  north  by  the  Sandia  Military  Reservation, 
which  includes  Kirtland  Air  Force  Base,  the  Manzano  Mountains  on  the  east,  and  the  Rio  Puerco 
and  Laguna  Pueblo  Reservation  on  the  west.”  7  The  area  referred  to  as  Site  B  in  the  Draft  Site 
Assessment  Report,17  which  contains  target  SI,  comprises  an  area  of  approximately  7000  acres 
that  were  leased  from  the  Tribe  in  the  1950’s  for  use  as  a  target  bombing  range  for  aircraft  from 
Kirtland.  Documentation  in  Bureau  of  Indian  Affairs  files  indicate  that  this  area  was  used  as  a 
practice  bombing  range  from  1956  to  1961  to  determine  the  performance  of  fast  aircraft  during 
bombing  runs.  In  the  1960’s,  Kirtland  collected  and  piled  visible  ordnance  debris  for  removal. 
Up  to  2  tons  of  practice  bombs  and  ordnance  waste  per  acre  were  removed  but  no  explosive 
ordnance  was  found. 

3.3.1  Climate  and  Weather 

During  the  month  of  February,  the  normal  high  temperature  in  Albuquerque  is  53  °F  with  a 
nonnal  low  of  26  F.  The  average  temperature  is  41  °F.  Of  more  importance  for  survey  work, 
February  is  the  second  driest  month  historically  with  normal  precipitation  of  just  under  0.5 
inches.  In  February  2002,  the  mean  wind  was  less  than  2  mph  from  the  SW. 

The  conditions  during  2003  were  less  benign.  Los  Lunas,  the  reporting  station  nearest  the  site, 
received  nearly  three  times  the  historical  mean  rain  during  February  2003.  This  complicated 
delivery  of  our  survey  equipment  and  supplies  to  the  site.  After  one  particularly  hard  rain,  road 
conditions  prevented  MTADS  personnel  from  reaching  the  site.  During  the  entire  month,  the 
area  had  a  more  active  weather  pattern  than  usual  resulting  in  several  periods  in  which  the  winds 
were  too  high  for  low-level  airborne  surveys  as  will  be  detailed  in  the  survey  log  below. 

3.3.2  Topography 

The  site  consists  of  open,  semi-arid  terrain.  The  area  is  relatively  flat,  open  grassland  with 
elevation  increasing  from  5100  feet  above  sea  level  on  the  west  to  a  maximum  of  5400  feet 
above  sea  level  and  a  broken  escarpment  on  the  east. 

3.3.3  Site  Maps  and  Photographs 

Figure  7  is  a  portion  of  a  USGS  7.5-minute  topo  map  showing  the  location  of  Target  SI  with  the 
approximate  boundaries  of  the  proposed  surveys  outlined.  The  most  direct  access  to  the  site  is 
by  a  dirt  road  that  exits  to  the  north  from  New  Mexico  Highway  6,  14  miles  west  of  Exit  203  off 
Interstate  25.  This  dirt  road  is  maintained  by  Isleta  Pueblo  personnel.  An  NRL  contractor, 
Geometries  GPS,  Inc.  of  Fredericksburg,  VA,  has  established  two  geodetic  survey  points  near 
Target  SI  and  four  more  on  various  other  targets  on  the  Pueblo’s  Comanche  Ranch.  The 
approximate  positions  of  the  two  first-order  points  near  Si  are  indicated  in  Figure  7.  The 
coordinates  of  all  six  points  are  given  in  Table  1.  An  example  of  the  predominant  M38s  and  one 
of  the  few  heavy- walled  fragments  is  shown  in  Figure  8. 
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UTM  Northing  (m) 

3,855,000  3,856,000  3,857,000  3,858,000  3,859,000  3,860,000 


318,000  319,000  320,000  321,000 


Fig.  7  -  A  portion  of  a  USGS  topo  map  showing  the  boundaries  of  the  planned  survey  areas.  The  locations  of 
the  two  first-order  points  installed  for  this  survey  are  shown  as  stars. 
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Table  1.  Isleta  Survey  Coordinates  Installed  for  this  Demonstration 


Point 

Latitude 

Longitude 

Northing  (m) 

Easting  (m) 

Ellipsoid 
Height  (m) 

NAD  83 

1A 

34°  50’  09.53499"  N 

106°  59’  12.69597"  W 

3,856,654.157 

318,321.948 

1528.443 

IB 

34°  51’  12.19331"  N 

106°  59’  18.29422"  W 

3,858,587.492 

318,218.027 

1541.863 

2 

34°  41’ 21.33042"  N 

106°  54’ 36.41382"  W 

3,840,244.133 

325,030.974 

1486.639 

3 

34°  33’  12.69605"  N 

106°  56’  50.72926"  W 

3,825,255.338 

321,322.056 

1535.667 

7 

34°  31’ 20.82374"  N 

107°  03’ 41.28845"  W 

3,822,016.365 

310,786.481 

1616.955 

8 

34°  40’  03.72964"  N 

107°  05’  21.49078"  W 

3,838,179.459 

308,565.015 

1702.621 

Fig.  8  -  Photograph  of  the  M38  practice  bombs  (left  panel)  and  one  of  the  few  heavy-walled  fragments 
(right  panel)  found  on  the  site 


3.4  Testing  and  Evaluation  Plan 

3.4.1  Pre-Demonstration  Activities 

The  MTADS  vehicular  system  as  well  as  components  of  the  airborne  system  were  mobilized  to 
the  Isleta  Pueblo  S-l  site  in  a  rented  53-ft  trailer.  The  tow  vehicle,  the  magnetometer  trailer, 
notebook  computers  for  the  DAS  and  Oasis  rnontaj™,  an  office  PC,  GPS  equipment,  batteries 
and  chargers,  office  equipment,  radios  and  chargers,  tools,  equipment  spares,  and  maintenance 
items,  and  the  airborne  boom  components  and  magnetometers  were  transported  in  the  trailer.  A 
government  contract  transportation  firm  transported  the  trailer  to  the  site.  The  helicopter  was 
mobilized  to  the  site  by  the  helicopter  charter  firm,  Helicopter  Transport  Services. 

Due  to  the  remoteness  of  the  survey  site,  no  essential  support  services  were  available  on-site. 
Accordingly,  NRL  made  provisions  to  acquire  all  of  the  requisite  supplies,  materials,  and 
facilities  from  rental  firms  in  Albuquerque.  For  this  operation  one  trailer  was  used  exclusively 
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for  data  processing  and  analysis,  as  a  communications  center,  battery  storage  and  charging 
stations,  an  electronics  repair  station,  and  storage  for  spares  and  supplies.  This  trailer  had  AC 
power  and  heat.  A  second  8  foot  x  48  foot  trailer,  which  could  be  fully  opened  from  either  end 
(for  drive  through),  was  used  to  garage  and  for  secure  storage  of  the  MTADS  vehicle  and  sensor 
platform.  Power  to  the  trailers  was  provided  by  a  65  KW  diesel  field  generator  that  was  also 
used  to  recharge  the  vehicle,  radios  and  GPS  batteries  overnight.  Communications  among  on¬ 
site  personnel  was  provided  by  hand-held  VHF  radios,  with  a  base  station  located  in  the 
command  trailer.  Radios  were  provided  to  all  field  and  office  teams.  Cellular  phone  service  was 
sporadically  available  at  the  office  trailer  with  more  reliable  service  in  the  hills  to  the  east  or  on 
the  state  highway.  Fuel  storage  was  provided  for  the  AC  generator  and  two  portable  toilets  were 
provided  to  support  all  field  and  office  crews  with  weekly  servicing.  Figure  9  shows  the 
arrangement  of  the  MTADS  base  camp  for  this  survey.  Aviation  fuel  to  support  the  airborne 
survey  was  also  located  on-site  although  at  some  distance  from  the  trailer  area.  The  combination 
of  an  on-site  fuel  supply  and  our  ability  to  base  the  helicopter  at  a  nearby  community  airport 
(Belen,  NM)  allowed  us  to  complete  the  survey  with  only  4.1  hrs  non-survey  helicopter  time. 


Fig.  9  -  Photograph  of  the  MTADS  base  camp  for  this  survey 
showing  the  office  and  garage  trailers,  generator,  diesel  tank, 
and  transport  trailer.  The  aviation  fuel  depot  is  not  shown  in 
this  photo. 

The  area  around  Target  SI  was  divided  into  two  survey  sites  in  the  planning  stages  of  this 
Demonstration.  A  larger,  1500-acre,  site  was  designated  for  the  two  airborne  systems.  Within 
this  site,  a  second,  100-acre  site  contained  the  seed  ordnance  and  was  to  be  surveyed  by  the 
vehicular  system.  The  coordinates  for  the  both  areas  are  given  in  Table  2. 


Table  2.  Coordinates  for  the  Corners  of  the  Two  Surveys 


Point 

Latitude 

Longitude 

Northing  (m) 

Easting  (m) 

NAD  83 

Air-NW 

34°  51'  42.726"N 

106°  59’  31.494"W 

3,859,534.87 

317,901.48 

Air-NE 

34°  51'  42.972"N 

106°  58’  08.556”W 

3,859,500.82 

320,007.88 

Air-SE 

34°  50’  09.696"N 

106°  58’  08.724”W 

3,856,627.06 

319,947.15 

Air-SW 

34°  50’  09.576"N 

106°  59’  31.632”W 

3,856,664.97 

317,840.93 
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Point 

Latitude 

Longitude 

Northing  (m) 

Easting  (m) 

NAD  83 

Vehicle-NW 

34°  51’  18.912"N 

106°  59’  05.400”W 

3,858,788.00 

318,549.62 

Vehicle  -NE 

34°  51’  19.038”N 

106°  58’  55.650”W 

3,858,786.99 

318,797.32 

Vehicle  -SE 

34°  50’  26.694”N 

106°  58’  56.400”W 

3,857,174.63 

318,746.38 

Vehicle  -SW 

34°  50’  26.940”N 

106°  59’  06.294”W 

3,857,187.19 

318,495.20 

3.4.2  Period  of  Operation 

The  NRL  portion  of  the  demonstration  survey  was  accomplished  from  Wednesday,  February  19th 
through  Thursday,  February  27th.  The  start  of  the  survey  was  delayed  two  days  from  the  planned 
start  due  to  the  mid-  February  snow  on  the  East  Coast  that  closed  all  area  airports  for  several 
days.  The  vehicular  survey  was  terminated  one  day  earlier  than  planned  because  of  a  terminal 
equipment  failure.  The  details  of  the  airborne  survey  are  given  in  Table  3  and  the  vehicular 
survey  in  Table  4. 

For  bookkeeping  convenience,  the  original  airborne  survey  area  was  divided  into  12  sorties  of  25 
survey  lines  each  (175m  east  to  west).  These  planned  sorties  and  their  relation  to  the  vehicular 
site  are  shown  schematically  in  Figure  10.  An  additional  sortie  was  added  to  the  west  side  of  the 
survey  when  the  eastern  edge  proved  too  hilly  and  tree-covered  for  efficient  surveying. 


3860000 


3859000 


O) 

c 


3858000 


I— 

Z) 


3857000 


3856000 


Fig.  10  -  Layout  of  the  12  planned  airborne  sorties 
(blue  lines)  showing  their  relation  to  the  vehicular 
area  (black  lines) 
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Table  3.  Details  of  the  Airborne  Survey 


Date 

Activity 

Survey 
Base  Name 

Duration 

(min.) 

Wed.,  Feb  19th 

MTADS  personnel  arrive  at  site.  Unpack  trailer 
and  set  up  office.  Transport  airborne  components 
to  Belen,  NM  airport  and  assemble  sensor  boom. 

Thurs,  Feb  20th 

Aircraft  arrives  ABQ  late  afternoon.  Assemble 
boom  supports  to  aircraft. 

Fri,  Feb  21st 

Ferry  aircraft  to  Belen.  Installation  complete  by 
10:30.  High  winds  prevent  survey.  Test  flight 
conducted  late  in  the  day. 

03053004 

14 

Replace  Mag  Sensor  #6 

Survey  Tracks  1-15  of  sortie  7 

03054001 

49 

30354002 

34 

Sat,  Feb  22nd 

Survey  Vehicular  Site  (Tracks  23-25  of  sortie  7 
and  all  of  sortie  8) 

30354003 

30354004 

56 

59 

03054005 

51 

03355003 

61 

Sortie  9 

03355004 

20 

03055005 

60 

Tracks  15-23  of  sortie  7 

03055006 

51 

Sun,  Feb  23rd 

Test  flight  for  eastern  edge  of  site.  Track  1  of 
both  sorties  1  and  2 

03055007 

14 

03055008 

44 

Sortie  3 

03055009 

42 

Tracks  1-17  of  sortie  4 

03055010 

03055011 

44 

24 

03055012 

57 

Tracks  15-25  of  sortie  4 

03056001 

47 

03056002 

45 

Sortie  5 

03056003 

43 

Mon,  Feb  24th 

03056004 

29 

03056005 

61 

Sortie  6 

03056006 

19 

03056007 

37 

Tracks  1  -  10  of  sortie  10 

03056008 

47 
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Date 

Activity 

Survey 
Base  Name 

Duration 

(min.) 

Tracks  8  -  25  of  sortie  10 

03057001 

03057002 

60 

17 

03057003 

39 

Sortie  1 1 

03057004 

45 

03057005 

28 

03057006 

43 

Tue,  Feb  25th 

Sortie  12 

03057007 

50 

03057008 

23 

03057009 

45 

Sortie  13 

03057010 

9 

03057011 

45 

03057012 

34 

Re-survey  tracks  1 1  and  12  of  sortie  3 

Remove  equipment  from  aircraft 

Aircraft  departs  site  for  ABQ  and  ferry  home 

03057013 

15 

Total  survey  time  minus  test  flight 

24  hrs  7  min 

Table  4.  Details  of  the  Vehicular  Survey 


Date 

Activity 

Survey 
Base  Name 

Duration 

(min.) 

Mon,  Feb  24th 

Static  Test 

03055001 

26 

Site  Survey 

03055002 

55 

03056001 

31 

03056002 

28 

Site  Survey 

03056003 

58 

03056004 

60 

Tue,  Feb  25th 

03056005 

58 

Calibration  Area 

03056006 

61 

03056007 

17 

Site  Survey 

03056008 

55 

03056009 

58 

03057001 

58 

Wed,  Feb  26th 

Site  Survey 

03057002 

19 

03057003 

59 

18 


Date 

Activity 

Survey 
Base  Name 

Duration 

(min.) 

03057004 

50 

03057005 

29 

03057007 

58 

Wed,  Feb  26th 

03057008 

15 

(continued) 

03057009 

57 

03057010 

31 

Calibration  Area  infill 

03057011 

2 

03056004  infill 

03057012 

4 

03058001 

60 

03058003 

61 

03058004 

61 

Thurs,  Feb  27th 

Site  Survey 

03058005 

63 

03058006 

58 

03058007 

52 

03058008 

30 

Sensor  boom  delaminates,  survey  tenninated 

Fri,  Feb  28th 

Pack  equipment  for  shipment. 

MTADS  personnel  depart  site. 

Total  survey  time  minus  static  test 

20  hrs  48  min 

3.4.3  Area  Characterized 

The  vehicular  MTADS  system  covered  28.1  hectares  (69.5  acres)  including  a  10-m  buffer 
around  the  vehicular  site,  Figure  11.  In  this  area,  we  marked  1364  targets  (16  of  these  were  the 
calibration  targets  planted  in  the  upper  part  of  the  site)  for  remediation  using  the  now  familiar 
classification  scheme  of  1  for  high-confidence  ordnance,  2  for  medium-confidence  ordnance,  3 
for  low-confidence  ordnance,  4  for  low-confidence  clutter,  5  for  medium-confidence  clutter,  and 
6  for  high-confidence  clutter.  A  breakdown  of  the  distribution  of  these  picks  is  given  in  Table  5. 
An  example  page  of  the  vehicular  target  list  is  in  Table  6  and  the  entire  list  is  included  as 
Appendix  A. 


Table  5.  Vehicular  MTADS  Target  Picks  Reported  by  Analyst’s  Classification 


UXO  Classification 

Cal 

1 

2 

3 

4 

5 

6 

Total 

Number  of  Picks 

16 

305 

328 

322 

239 

137 

17 

1364 

19 


Table  6.  A  Sample  of  the  MTADS  Vehicular  Target  Report  for  the  Vehicular  Area 


Target 

ID 

UTM  E  (m) 

UTM  N  (m) 

Depth 

(m) 

Size 

(m) 

Moment 

Incl. 

Azim. 

Fit 

Quality 

Analyst's  Comments 

uxo 

Category 

Latitude 

Longitude 

1 

318557.47 

3858778.94 

0.37 

0.053 

0.078 

51 

75 

0.943 

81  mm  mortar  -  serial  #206 

cal 

34.855173011 

-106.984745505 

2 

318570.85 

3858778.12 

0.34 

0.079 

0.257 

59 

54 

0.984 

105mm  proj  -  serial  #  208 

cal 

34.855168015 

-106.984599028 

3 

318582.15 

3858774.76 

0.26 

0.038 

0.028 

62 

3 

0.945 

60mm  mortar  -  serial  #  60 

cal 

34.855139755 

-106.984474827 

4 

318592.92 

3858777.26 

0.35 

0.128 

1.074 

44 

84 

0.989 

105mm  mortar  -  serial  #  209 

cal 

34.855164212 

-106.984357519 

5 

318603.49 

3858777.82 

0.26 

0.210 

4.810 

3 

93 

0.998 

2.75in  rocket  -  serial  #810 

cal 

34.855171164 

-106.984242144 

6 

318644.95 

3858773.30 

0.43 

0.075 

0.217 

62 

92 

0.987 

81  mm  mortar  -  serial  #  207 

cal 

34.855137846 

-106.983787876 

7 

318659.24 

3858774.39 

0.44 

0.093 

0.416 

37 

5 

0.986 

81  mm  mortar  -  serial  #  208 

cal 

34.855150221 

-106.983631930 

8 

318673.61 

3858773.72 

0.42 

0.151 

1.766 

30 

14 

0.993 

105mm  proj  -  serial  #  21 1 

cal 

34.855146722 

-106.983474620 

9 

318687.04 

3858775.51 

0.22 

0.061 

0.119 

35 

303 

0.947 

60mm  mortar  -  serial  #  64 

cal 

34.855165222 

-106.983328224 

10 

318699.22 

3858772.76 

0.38 

0.245 

7.559 

23 

92 

0.975 

2.75in  rocket  -  serial  #  803 

cal 

34.855142648 

-106.983194440 

11 

318720.48 

3858772.06 

0.34 

0.123 

0.968 

15 

9 

0.994 

105mm  proj  -  serial  #213 

cal 

34.855140113 

-106.982961845 

12 

318736.84 

3858771.62 

0.37 

0.071 

0.182 

30 

21 

0.972 

81  mm  mortar  -  serial  #  209 

cal 

34.855139024 

-106.982782882 

13 

318756.48 

3858773.67 

0.22 

0.078 

0.249 

4 

187 

0.984 

60mm  mortar  -  serial  #  65 

cal 

34.855161006 

-106.982568588 

14 

318774.03 

3858772.50 

0.35 

0.232 

6.482 

-25 

184 

0.989 

2.75in  rocket  -  serial  #  805 

cal 

34.855153671 

-106.982376513 

15 

318797.59 

3858771.99 

0.30 

0.261 

9.161 

10 

7 

0.968 

2.75in  rocket  -  serial  #  800 

cal 

34.855153209 

-106.982118782 

16 

318623.26 

3858773.64 

0.22 

0.051 

0.070 

20 

22 

0.976 

60mm  mortar  -  serial  #  77 

cal 

34.855137015 

-106.984025094 

17 

318510.16 

3857181.40 

0.63 

0.193 

3.688 

38 

23 

0.995 

M38 

1 

34.840767196 

-106.984916880 

18 

318532.18 

3857177.87 

1.77 

0.201 

4.172 

86 

153 

0.922 

M38  outside  boundary 

1 

34.840739298 

-106.984675387 

19 

318557.25 

3857173.06 

0.49 

0.044 

0.045 

18 

38 

0.891 

possible  60mm  outside 
boundary 

3 

34.840700457 

-106.984400351 

20 

318573.86 

3857183.62 

0.30 

0.043 

0.042 

9 

35 

0.896 

possible  60mm 

3 

34.840798531 

-106.984220994 

21 

318586.79 

3857181.62 

0.13 

0.058 

0.103 

9 

271 

0.971 

remanent 

4 

34.840782789 

-106.984079308 

22 

318585.28 

3857175.08 

0.46 

0.090 

0.378 

23 

13 

0.986 

M38  oustide  boundary 

2 

34.840723614 

-106.984094367 

23 

318583.09 

3857173.24 

0.00 

0.040 

0.032 

32 

355 

0.985 

possible  60mm  outside 
boundary 

1 

34.840706666 

-106.984117938 

24 

318580.24 

3857173.69 

0.28 

0.040 

0.032 

16 

132 

0.883 

clutter 

4 

34.840710206 

-106.984149201 

25 

318581.22 

3857175.07 

0.05 

0.027 

0.010 

-14 

47 

0.913 

small  clutter 

4 

34.840722807 

-106.984138692 

26 

318607.15 

3857182.09 

0.93 

0.257 

8.735 

-63 

28 

0.779 

large  clutter 

6 

34.840790716 

-106.983856803 

27 

318630.12 

3857184.76 

0.06 

0.032 

0.016 

-42 

4 

0.949 

small  clutter 

4 

34.840818818 

-106.983606348 

28 

318646.91 

3857179.96 

0.15 

0.030 

0.014 

33 

323 

0.926 

small  clutter 

4 

34.840778574 

-106.983421710 

29 

318658.25 

3857171.76 

0.50 

0.338 

20.012 

26 

4 

0.966 

GP  bomb  outside  area 

2 

34.840706712 

-106.983296033 

30 

318654.82 

3857177.09 

0.29 

0.058 

0.099 

57 

118 

0.942 

possible  60mm  outside 
boundary 

2 

34.840754176 

-106.983334649 

UTM  Northing  (m) 


318,000  319,000  320,000 


UTM  Easting  (m) 

Fig.  1 1  -  Overview  of  target  SI  showing  the  portion  of  the  site  covered  by  the  vehicular  system 
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The  airborne  MTADS  system  surveyed  570  hectares  (1408  acres),  Figure  12.  As  mentioned 
above,  the  terrain  and  tree  cover  on  the  two  eastern-most  sorties  would  have  required  a  survey  at 
greater  than  three  meters  above  the  ground.  This  would  greatly  compromise  our  ability  to  detect 
the  M38  and  BDU-33  ordnance  that  was  the  target  of  this  survey.  As  we  were  flying  our  survey, 
the  Oak  Ridge  group  was  finishing  up  by  flying  this  eastern  area  above  the  trees.  In  order  to 
maximize  the  survey  data  that  would  be  useful  for  the  Pueblo,  we  removed  sorties  1  and  2  (see 
Figure  10)  from  the  flight  list  and  added  a  new  sortie  13  on  the  western  edge  of  the  site.  This 
allowed  us  to  cover  almost  to  the  western  edge  of  the  Pueblo’s  land  around  target  SI. 

Airborne  targets  were  picked  in  two  areas.  The  first  area  was  the  100-acre  vehicular  site 
excluding  the  densest  target  area  surrounding  the  bull’s  eye.  These  targets  were  picked  and  the 
target  list  submitted  to  ESTCP  before  the  vehicular  survey.  Later,  the  ESTCP  Program  Office 
requested  that  the  airborne  analyst  pick  targets  closer  to  the  bull’s  eye.  In  response  to  this 
request  the  airborne  analyst,  who  was  not  on  site  during  the  vehicular  data  collection  and  had  no 
access  to  the  vehicular  data,  expanded  the  portion  of  the  100-acre  site  analyzed.  This  resulted  in 
1260  targets,  which  are  detailed  in  Appendix  B  and  categorized  in  Table  7.  An  example  of  the 
target  report  is  given  in  Table  8. 

The  second  area  in  which  targets  were  picked  was  designated  the  “Primary  Area.”  This  area  was 
chosen  by  the  ESTCP  Program  Office  in  conjunction  with  the  two  survey  teams.  Target  analysis 
in  this  area  resulted  in  388  targets,  which  are  detailed  in  Appendix  C  and  categorized  in  Table  7. 


Table  7.  Airborne  MTADS  Target  Picks  Reported  by  Analyst’s  Classification 


UXO  Classification 

Cal 

1 

2 

3 

4 

5 

6 

Total 

Vehicular  Area  Picks 

12 

502 

336 

282 

42 

69 

17 

1260 

“Primary  Area”  Picks 

93 

85 

70 

48 

52 

40 

388 
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Fig.  12  -  Overview  of  target  SI  showing  the  portion  of  the  site  covered  by  the  airborne  system 
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Table  8.  A  Sample  of  the  MTADS  Airborne  Target  Report  for  the  Vehicular  Area 


Targ. 

ID 

UTM  E  (m) 

UTM  N  (m) 

WjJWU 

■ESI 

Moment 

Incl. 

Azim. 

■ 

Analyst  Comments 

uxo 

Cat. 

Latitude 

Longitude 

1 

318594.70 

3858778.90 

Cal  Target  #4?  -  105mm 

Mortar  @  0.25m 

cal 

34.85517934 

-106.98433847 

2 

318603.44 

3858777.87 

1549.89 

0.20 

0.205 

4.4610 

6 

94 

0.988 

Cal  Target  #5?  -  2.75"  Rocket 
@  0.20m 

cal 

34.85517160 

-106.98424267 

3 

318644.65 

3858773.27 

1550.49 

0.17 

0.056 

0.0892 

85 

307 

0.865 

Cal  Target  #7?  -  81mm  Mortar 
@  0.25m 

cal 

34.85513747 

-106.98379114 

D 

318658.99 

3858774.45 

1552.06 

0.00 

0.038 

0.0294 

45 

338 

0.301 

Cal  Target  #8?  -  81mm  Mortar 
@  0.25m 

cal 

34.85515070 

-106.98363466 

5 

318673.60 

3858773.93 

1550.94 

0.32 

0.133 

1.2059 

20 

7 

0.946 

Cal  Target  #9?  -  105mm  Proj 
@  0.25m 

cal 

34.85514862 

-106.98347478 

6 

318699.86 

3858772.69 

1551.85 

0.14 

0.250 

8.0344 

22 

98 

0.975 

Cal  Target  #1 1  -  2.75"  Rocket 
@  0.20m 

cal 

34.85514213 

-106.98318742 

7 

318720.34 

3858772.12 

1551.42 

1.07 

0.176 

2.8408 

12 

354 

0.938 

Cal  Target  #12?  -  105mm  Proj 
@  0.25m 

cal 

34.85514062 

-106.98296341 

8 

318736.36 

3858771.76 

1551.31 

1.40 

0.126 

1.0376 

14 

350 

0.897 

Cal  Target  #  13?  -  81mm 

Mortar  @  0.25m 

cal 

34.85514024 

-106.98278812 

9 

318758.16 

3858775.04 

Cal  Target  #  14?  -  60mm 

Mortar  @  0.15m 

cal 

34.85517372 

-106.98255050 

10 

318774.03 

3858772.62 

1553.20 

0.52 

0.238 

6.9617 

-19 

187 

0.982 

Cal  Target  #15?  -  2.75"  Rocket 
@  0.20m 

cal 

34.85515469 

-106.98237646 

11 

318797.63 

3858772.00 

1554.08 

0.34 

0.224 

5.7912 

6 

5 

0.971 

Cal  Target  #16?  -  2.75"  Rocket 
@  0.20m 

cal 

34.85515333 

-106.98211834 

12 

318585.47 

3857174.90 

1534.97 

0.28 

0.076 

0.2307 

38 

29 

0.920 

81mm  9m  S  of  the  border 

2 

34.84072204 

-106.98409228 

13 

318588.04 

3857181.85 

1534.34 

1.17 

0.087 

0.3419 

50 

277 

0.737 

81mm  in  geology 

3 

34.84078513 

-106.98406561 

14 

318606.65 

3857181.68 

1535.62 

0.22 

0.201 

4.1988 

78 

278 

0.959 

M38,  SHALLOW 

1 

34.84078696 

-106.98386216 

15 

318609.22 

3857184.72 

1535.42 

0.70 

0.078 

0.2459 

0 

236 

0.628 

81MM  IN  CLUTTER 

3 

34.84081478 

-106.98383480 

16 

318614.19 

3857186.97 

1535.23 

0.89 

0.082 

0.2889 

24 

10 

0.669 

UNLIKELY  DEEP  81MM 

3 

34.84083593 

-106.98378097 

17 

318604.55 

3857203.14 

1535.49 

0.49 

0.106 

0.6151 

32 

16 

0.916 

105MM 

1 

34.84097995 

-106.98388979 

18 

318645.96 

3857165.26 

1535.65 

1.14 

0.198 

3.9924 

16 

356 

0.909 

LARGE  FOR  M3815M  S  OF 
BORDER 

1 

34.84064591 

-106.98342892 

19 

318658.29 

3857171.98 

1535.88 

0.67 

0.320 

16.8804 

17 

3 

0.980 

BOMB,  7  M  SOUTH  OF 
BORDER 

1 

34.84070866 

-106.98329563 

20 

318669.12 

3857174.00 

1535.11 

1.62 

0.142 

1.4926 

23 

164 

0.885 

DEEP  M38 

2 

34.84072887 

-106.98317770 

24 


3.4.4  Area  Remediated 


Targets  were  remediated  in  the  two  airborne  areas  mentioned  above,  the  Vehicular  Area  and  the 
Primary  Area.  The  coordinates  of  these  areas  are  listed  in  Table  9  and  their  relative  location  is 
shown  in  Figure  13.  A  limited  number  of  targets  were  also  remediated  at  sites  S2  and  S7  in 
support  of  an  earlier  Oak  Ridge  survey  of  those  sites.  Their  approximate  location  is  shown  in 
Figure  14. 


Table  9.  Coordinates  for  the  Corners  of  the  Two  Remediation  Areas  at  SI 


Point 

Latitude 

Longitude 

Northing  (m) 

Easting  (m) 

NAD  83 

Vehicle-NW 

34°  51'  18.912"N 

106°  59’  05.400"W 

3,858,788.00 

318,549.62 

Vehicle  -NE 

34°  51'  19.038"N 

106°  58’  55.650"W 

3,858,786.99 

318,797.32 

Vehicle  -SE 

34°  50’  26.694"N 

106°  58’  56.400"W 

3,857,174.63 

318,746.38 

Vehicle  -SW 

34°  50’  26.940”N 

106°  59’  06.294"W 

3,857,187.19 

318,495.20 

Primary-NW 

34°  51'  41.071”N 

106°  59’  27.914”W 

3,859,482.06 

317,991.39 

Primary  -NE 

34°  51'  41.420"N 

106°  59’  06.552"W 

3,859,482.06 

318,534.10 

Primary  -SE 

34°  51'  08.891"N 

106°  59’  05.770"W 

3,858,479.46 

318,534.10 

Primary  -SW 

34°  51’08.542"N 

106°  59’  27.130”W 

3,858,479.46 

317,991.39 

25 


UTM  Northing  (m) 


318,000  319,000  320,000 
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UTM  Northing  (m) 


310,000  320,000 


UTM  Easting  (m) 

Fig.  14  -  Map  of  the  region  showing  the  location  of  targets  S2  and  S7  relative  to  SI 
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3.4.5  Demobilization 


At  the  conclusion  of  the  survey,  all  MTADS  equipment  was  returned  to  Blossom  Point  in  the 
rented  53’  trailer  by  a  government  contract  transportation  firm.  The  helicopter  used  for  the 
airborne  portion  of  the  survey  was  ferried  back  to  its  base  in  Baltimore,  MD  by  the  charter 
company,  Helicopter  Transport  Services. 

The  remediated  targets  that  were  judged  to  be  dangerous  by  the  UXO  crews  were  blown  in  place. 
All  other  remediation  scrap  was  marked  as  inert  and  removed  by  a  commercial  scrap  hauler 
under  contract  to  EOTI,  Inc. 
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4.  Performance  Assessment 


4.1  Performance  Criteria 


Table  10.  Performance  Criteria  for  this  Demonstration 


Performance 

Criterion 

Description 

Primary  or 
Secondary 

Probability  of 
Detection  (Pd) 

Fraction  of  emplaced  targets  detected  by  the  technology. 

Primary 

False  Alarm 
Rate  (ha'1) 

Number  of  anomalies  per  hectare  not  corresponding  to 
ordnance  items  or  ordnance-related  material. 

Primary 

Location 

Accuracy 

Mean  miss  distance  for  successfully  detected  targets 

Primary 

Production 

Rate 

Area  surveyed  per  day  (or  hour)  for  each  technology 

Primary 

Ease  of  Use 

A  minimum  of  four  people  is  required  to  conduct  an  airborne 
MTADS  survey  with  an  additional  analyst  in  the  field,  or  at 
home,  to  complete  target  analysis.  They  include  a  site/project 
supervisor,  a  pilot,  a  data  acquisition  operator,  and  a  data 
preprocessor.  While  the  position  of  project  supervisor  and  data 
acquisition  operator  do  not  strictly  require  advanced  training, 
we  have  found  that  the  decision  and  diagnostic  skills  of  more 
highly-trained  scientists  and  engineers  result  in  project 
efficiencies  that  more  than  compensate  for  the  added  personnel 
cost.  At  the  present  stage  of  development,  the  data 
preprocessor  and  analyst  must  be  experts.  As  we  gain  more 
experience  with  the  methods  and  refine  the  default  assumptions 
we  expect  this  requirement  to  relax. 

Secondary 

Vehicular  surveys  are  typically  conducted  with  a  team  of  three 
NRL  personnel,  a  site/project  supervisor  who  assist  with  data 
preprocessing,  a  vehicle  operator,  and  an  on-site  data  analyst. 

The  data  flow  is  slower  with  the  vehicular  system  so  one  less 
analyst  is  required.  In  addition  to  these  positions,  we  typically 
employ  two  or  three  (two  in  this  case)  local  field  hands  to  assist 
with  vehicle  guidance,  maintenance,  and  operation. 
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Performance 

Criterion 

Description 

Primary  or 
Secondary 

Maintenance 

The  maintenance  required  for  MTADS  is  typical  of  that 
required  by  computers  and  peripherals,  laboratory  and  field 
electronics,  and  for  transportation  vehicles.  Maintenance 
Manuals  and  preventive  maintenance  procedures  have  been 
established  for  all  MTADS  subsystems.  Electronic  and 
mechanical  repair  equipment  and  tool  sets  are  packaged  for 
transport  to  all  demonstration  sites.  A  list  of  critical  spare 
components  is  maintained,  as  are  materials  for  anticipated 
maintenance  and  repairs  (vehicle  service,  cable  repairs, 
computer  change-out,  etc).  We  encountered  an  unusual  need 
during  this  survey,  fiberglass  repair.  We  are  not  equipped  to 
handle  this  in  the  field  so  we  truncated  the  survey  and  repaired 
the  damage  at  home. 

Secondary 

4.2  Performance  Confirmation  Methods 

Performance  of  each  of  the  two  systems  was  measured  in  two  ways.  First,  each  system  surveyed 
the  portion  of  the  site  in  which  the  ESTCP  Program  Office  arranged  for  a  variety  of  inert 
ordnance  to  be  seeded.  After  analysis  of  the  survey  data,  we  submitted  two  target  lists  to  the 
Institute  for  Defense  Analyses  (IDA)  for  scoring.  Using  the  known  positions  of  these  emplaced 
targets,  IDA  was  able  to  detennine  the  detection  performance  of  each  of  the  systems  for  the 
emplaced  items,  the  number  of  false  alarms,  and  the  location  accuracy  for  the  detected  items. 

Additionally,  690  (7 1 1  original  picks  -  2 1  inadvertently  included  seed  targets)  targets  were 
selected  by  IDA  and  the  ESTCP  Program  Office  for  remediation  by  UXO  technicians  from 
EOTI,  Inc.,  our  remediation  contractor.  Each  of  the  remediated  items  was  characterized, 
photographed,  and  located  by  GPS  re-survey.  From  these  data,  qualitative  conclusions  can  be 
drawn  about  each  systems  detection  performance  and  a  quantitative  measure  of  location  accuracy 
derived.  Each  of  these  two  sets  of  targets  are  discussed  separately  in  the  following  section. 

4.3  Data  Analysis,  Interpretation,  and  Evaluation 
4.3.1  Emplaced  Targets 

The  most  instructive  view  of  the  performance  of  the  two  systems  comes  when  considering  the 
emplaced  targets.  The  ESTCP  Program  Office  arranged  for  126  inert  UXO  items  to  be 
emplaced,  42  105-mm  projectiles,  16  2.75-in  warheads,  24  60-mm  mortars,  and  44  81-mm 
mortars.  The  location  of  four  of  each  ordnance  class  was  given  to  the  demonstrators  before  the 
survey,  these  served  as  calibration  targets.  Not  all  of  these  targets  are  appropriate  for  each  of  the 
platforms.  The  airborne  system  is  able  to  detect  the  mortars  only  under  the  most  favorable  noise 
conditions.  Any  statistics  on  detection  of  these  items  by  the  airborne  system  is  more  a  measure 
of  the  site  noise  than  the  sensitivity  of  the  system. 
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The  detection  perfonnance  and  false  alarm  rate  of  the  vehicular  system  with  respect  to  these 
targets  are  detailed  in  Table  11. 

Table  11.  Measured  perfonnance  against  the  emplaced  targets  for  the  vehicular  system 


Priority 

Alarm  rate 
(ha1)* 

Pd(overall) 

Pd(105) 

Pd(81) 

Pd(60) 

Pd(2.75) 

1.0  m 

1.5  m 

1.5  m 

1.5  m 

1.5  m 

1.5  m 

1 

1 

0.43 

0.43 

2 

3 

0.68 

0.68 

3 

7 

0.89 

0.89 

4 

12 

0.94 

0.94 

5 

15 

0.96 

0.96 

6 

16 

0.96 

0.96 

0.94 

0.95 

1.0 

1.0 

*number  of  picks  per  hectare  not  corresponding  to  emplaced  items  (cumulative) 


IDA  analysts  calculated  the  probability  of  detection  using  both  1.0  and  1.5-m  detection  radii.  As 
expected  from  our  past  measures  of  location  accuracy,  the  performance  of  the  vehicular  system 
was  independent  of  this  choice.  The  actual  location  performance  of  the  vehicular  system  is  given 
in  Table  12  which  shows  no  significant  bias  but  a  standard  deviation  slightly  larger  than  we 
would  have  expected. 


Table  12.  Location  performance  of  the  vehicular  system 
against  the  emplaced  targets 


A  Easting 

A  Northing 

Mean  (cm) 

-1 

4 

Std  Dev  (cm) 

12 

13 

It  is  interesting  to  note  that  the  vehicular  system  failed  to  detect  one  of  the  emplaced  105-mm 
projectiles  and  one  of  the  2.75-in  warheads,  nominally  the  easier  targets  to  detect.  A  detail  of  a 
small  portion  of  the  vehicular  survey  data  that  explains  these  missed  detections  is  shown  in 
Figure  15.  A  portion  of  one  survey  line,  which  coincidentally  contains  the  two  missed  targets,  is 
missing.  This  is  a  common  occurrence  and  usually  stems  from  poor  GPS  fit  quality  for  a  few 
seconds  to  minutes.  The  vehicular  data  analyst  flags  these  missing  lines  for  resurvey,  which  is 
usually  accomplished  at  the  end  of  the  day  while  the  field  crew  is  securing  the  site  for  the 
evening.  This  missing  line  was  marked  as  described  but  the  sensor  platform  failure  occurred 
before  the  resurvey  occurred. 

This  perfonnance  is  likely  typical  of  real-world  survey  conditions  where  small  data  gaps  are 
unavoidable  due  to  deep  ravines,  trees,  large  bushes,  etc.  It  unfortunately  demonstrates  the 
difficulties  associated  with  a  declaration  that  a  site  is  completely  clean. 
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Fig.  15  -  A  small  portion  of  survey  data  from  the  vehicular  system 
showing  a  short  segment  of  missing  data  that  contains  two  of  the 
emplaced  targets 


The  performance  of  the  airborne  system  to  these  same  targets  is  detailed  in  Table  13.  As  was  the 
case  with  the  vehicular  system,  there  is  a  negligible  difference  between  the  results  using  a  1 .0-m 
detection  radius  and  1.5  m.  We  have  demonstrated  previously  that  our  target  location  accuracy 
approaches  0.5  m  so  this  is  not  unexpected.  The  actual  location  accuracy  is  given  in  Table  14. 

Also  not  surprising  is  the  detection  probability  for  60-mm  mortars.  As  mentioned  above,  these 
targets  are  below  the  detection  threshold  of  the  airborne  system  except  in  the  quietest 
environments.  A  calculation  of  detection  performance  without  considering  the  60-mms  is  given 
in  Table  15. 


Table  13.  Measured  performance  against  the  emplaced  targets  for  the  airborne  system 


Priority 

Alarm  rate 
(ha1)* 

Pd(overall) 

Pd(105) 

Pc(81) 

Pd(60) 

Pd(2.75) 

1.0  m 

1.5  m 

1.5  m 

1.5  m 

1.5  m 

1.5  m 

1 

2 

0.21 

0.21 

2 

4 

0.38 

0.39 

3 

7 

0.51 

0.54 

4 

7 

0.52 

0.54 

5 

9 

0.53 

0.55 

6 

11 

0.54 

0.56 

0.73 

0.48 

0.20 

0.91 

*number  of  picks  per  hectare  not  corresponding  to  emplaced  items  (cumulative) 
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Table  14.  Location  performance  of  the  airborne  system 
against  the  emplaced  targets 


A  Easting 

A  Northing 

Mean  (cm) 

-4 

4 

Std  Dev  (cm) 

31 

30 

Table  15.  Measured  performance  for  the  airborne 
system  excluding  the  60mm  mortars 


Priority 

Alarm  rate 
(ha1)* 

Pd(overall) 

1.0  m 

1.5  m 

1 

2 

0.26 

0.26 

2 

4 

0.45 

0.46 

3 

7 

0.59 

0.61 

4 

7 

0.60 

0.62 

5 

9 

0.61 

0.63 

6 

11 

0.62 

0.64 

‘number  of  picks  per  hectare  not  corresponding 
to  emplaced  items  (cumulative). 


Even  without  the  mortars,  the  airborne  system  failed  to  correctly  report  13  of  the  105mm  and 
2.75in  targets.  Detail  of  the  analysis  of  these  targets  is  given  in  Table  16. 


Table  16.  Detail  of  the  larger  targets  not  reported  by  the  airborne  system 


Target  ID 

Weak 

Response 

Signal 

Overlap 

Analysis 

Error 

Comments 

105mm-02 

X 

North  of  Survey  Area 

105mm-03 

X 

North  of  Survey  Area 

105mm-06 

X 

Target  lost  in  geology 

105mm-07 

X 

4nT  signal,  not  picked 

105mm- 12 

X 

Target  lost  in  geology 

105mm- 16 

X 

No  measurable  signal 

105mm-21 

X 

1 .5 1-m  position  error 

105mm-25 

X 
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Target  ID 

Weak 

Response 

Signal 

Overlap 

Analysis 

Error 

Comments 

105mtn-28 

X 

Single  track  6  nT  peak  signal 

105mtn-37 

X 

Target  lost  in  geology 

105mtn-39 

X 

Target  lost  in  geology 

2.75in-02 

X 

Sensors  4.1m  AGL 

2.75in-04 

X 

No  measurable  signal 

The  detection  data  for  both  systems  are  shown  plotted  in  a  ROC  curve  in  figure  16.  Both 
systems  exhibit  reasonable  classification  performance  as  the  bulk  of  the  detections  (93%  for  the 
vehicular  system  and  95%  for  the  airborne)  occur  in  the  first  three  priority  groups,  high- 
confidence  ordnance,  medium-confidence  ordnance,  and  low-confidence  ordnance.  Most  of  the 
ordnance  targets  categorized  in  the  lower  three  groups  are  the  result  of  remenant  magnetization 
of  these  targets  as  illustrated  for  a  vehicular  anomaly  in  Figure  17.  Although  there  is  recent 

1  8 

evidence  that  using  the  presence  of  significant  remenant  magnetization  may  not  be  justified,  it 
is  a  standard  practice  and  was  used  by  both  analysis  teams. 


Fig.  16  -  Detection  data  for  the  emplaced  targets  for  both 
systems  plotted  as  an  ROC  curve 
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Fig.  17  -  Example  of  an  emplaced  target  with  significant 
remenant  magnetization  that  was  misprioritized  in  the 
vehicular  analysis 


4.3.2  Remediated  Targets  in  the  Vehicular  Area 

Of  the  371  targets  in  the  vehicular  area  submitted  to  the  remediation  contractor  for  investigation, 
338  were  actually  dug.  A  large  majority  of  the  remaining  33  targets  were  skipped  for  resource 
allocation  reasons.  At  the  end  of  the  first  week  of  remediation,  we  realized  that  targets  below  5 
feet  deep  were  consuming  an  inordinate  fraction  of  the  remediation  resources  available.  In  order 
to  maximize  the  number  of  targets  remediated,  and  thus  maximize  the  amount  of  comparison 
ground  truth,  we  moved  the  remaining  deep  targets  to  the  bottom  of  the  dig  priority  list.  The 
handful  of  missed  targets  that  were  not  deep  were  missed  due  to  clerical  errors  by  the 
remediation  teams  (target  removed  but  dig  sheet  not  filled  out,  numbers  transposed  in  the  target 
reports,  targets  incorrectly  checked  off  on  the  dig  list,  etc.).  The  dig  list  for  the  vehicular  area 
with  full  remediation  details  is  included  in  Appendix  D  with  the  missed  targets  highlighted. 

Just  as  in  the  case  of  the  emplaced  targets,  we  can  examine  the  performance  of  both  systems  with 
regard  to  the  targets  remediated  in  the  vehicular  area.  Figure  18  shows  the  location  performance 
of  the  two  systems  plotted  on  a  polar  plot.  The  vehicular  data  are  shown  in  the  left  panel  as  blue 
symbols  and  the  airborne  data  are  in  the  right  panel  as  red  symbols.  The  outer  radius  shown  in 
the  plots  corresponds  to  the  1.5-m  halo  used  by  IDA  for  their  analysis.  As  in  the  case  of  the 
emplaced  targets,  a  large  majority  of  the  target  picks  for  both  systems  is  within  the  more 
restrictive  1.0-m  halo.  The  vehicular  system  displays  a  tighter  clustering  around  the  measured 
location  of  the  remediated  targets  due  to  the  higher  data  density  of  the  vehicular  anomaly  maps 
and  may  have  a  slight  bias  to  the  northwest.  The  airborne  data  appear  to  be  symmetrically 
clustered  around  the  measured  positions  with  most  of  the  reported  positions  inside  a  0.5-m 
radius. 

Greater  detail  on  the  radial  miss  distance  is  given  in  Table  17  and  illustrated  in  Figure  19.  The 
location  perfonnance  against  these  targets  is  very  close  to  that  measured  against  the  emplaced 
targets. 


35 


Vehicular 

o° 


Airborne 


o° 


Fig.  18  -  Location  performance  of  the  two  systems  for  the  remediated  targets  in  the  vehicular  area.  The 
vehicular  data  are  shown  on  the  left  as  blue  symbols  and  the  airborne  data  are  on  the  right  as  red  symbols. 


Table  17.  Detail  of  the  location  performance  of  the  two  systems  against  the  remediated 
targets  in  the  vehicular  area. 


System 

Mean  Miss  Distance 

90%  Within 

95%  Within 

Vehicular 

35  cm 

59  cm 

77  cm 

Airborne 

49  cm 

90  cm 

108  cm 

Vehicular 


Airborne 


miss  distance  (m) 


miss  distance  (m) 


Fig.  19  -  Comparison  of  the  location  accuracy  of  the  two  systems  for  the  remediated  targets  in  the  vehicular 
area.  The  vehicular  data  are  plotted  on  the  left  and  the  airborne  on  the  right. 


36 


Again,  as  in  the  case  of  the  emplace  targets,  a  large  fraction  of  the  remediated  targets  that 
corresponded  to  ordnance  and  ordnance-related  fragments  and  scrap  were  categorized  in  the  first 
three  priority  groups.  This  is  shown  graphically  in  Figure  20  where  the  cumulative  fraction  of 
ordnance  related  targets  is  plotted  against  the  priority  group  number. 
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Fig.  20  -  Cumulative  fraction  of  ordnance-related  remediated  targets  on 
the  vehicular  site  by  priority  category  for  the  two  systems 


4.3.3  Remediated  Targets  in  the  Primary  Area 

Of  the  319  targets  selected  for  remediation  in  the  primary  area,  all  but  one  were  investigated. 
The  one  missed  target  was  listed  as  15.9  feet  deep  and  was  skipped  due  to  the  extreme  cost  of 
remediation.  A  dig  list  for  this  area  with  complete  remediation  results  is  included  in  Appendix 
E.  The  striking  feature  of  this  remediation  was  the  substantial  number  of  targets  for  which  no 
metal  object  was  found  by  the  remediation  team.  These  targets  are  listed  in  the  remediation 
results  as  “no  find”  or  “negative  find.” 

Since  empty  holes  were  not  a  problem  in  the  vehicular  area,  the  ESTCP  Program  Office  asked 
each  of  the  airborne  teams  to  submit  a  list  of  twenty  of  these  “no  finds”  that  they  would  like  to 
have  re-investigated.  Members  of  the  vehicular  MTADS  team  returned  to  the  site  during  the 
third  week  in  January  2004  to  reinvestigate  the  selected  targets. 

4.3.4  Reinvestigation  of  “No  Finds”  in  the  Primary  Area 

Each  of  the  two  airborne  analysis  teams  (NRL  and  ORNL)  chose  20  targets  from  the  list  of  “no 
finds”  for  reinvestigation  and  an  additional  10  as  backups.  On  January  18th  2004,  two  members 
of  the  vehicular  MTADS  team  redeployed  to  target  SI  to  reinvestigate  the  selected  targets  using 
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the  MTADS  man-portable  magnetometer  system  with  two  members  of  the  remediation  team 
arriving  the  next  day.  The  details  of  this  reinvestigation  are  given  in  Table  18. 


Table  18.  Details  of  the  Man-Portable  Survey 


Date 

Activity 

Survey  Base 
Names 

Sun,  Jan  18th 

MTADS  personnel  arrive  in  Albuquerque. 

Mon,  Jan  19th 

Pick  up  MTADS  equipment.  Meet  and  brief  Isleta 
Environmental  Office,  Assemble  equipment  and  survey  first 
ten  targets. 

04019003 

thru 

04019012 

Tue,  Jan  20th 

Mark  first  ten  remediation  locations. 

Survey  second  ten  targets. 

Work  day  cut  short  by  afternoon  rain 

Remediate  first  ten  targets. 

04020001 

thru 

04020010 

Wed,  Jan  2 1st 

Mark  next  ten  remediation  locations. 

Survey  sixteen  additional  targets. 

Mark  ten  additional  remediation  locations. 

Remediate  fifteen  targets. 

04021001 

thru 

04021014 

04021016 

04021017 

Thurs,  Jan  22nd 

Survey  eleven  targets. 

Mark  last  five  remediation  locations. 

Remediate  ten  targets. 

Fill  and  level  all  holes. 

Clean  area  of  stakes  and  flagging. 

04022001 

thru 

04022011 

Fri,  Jan  23rd 

Return  Borrowed  GPS  equipment  and  rental  vehicles. 
MTADS  personnel  depart  Albuquerque. 

Survey  scrap  certified  and  removed  from  the  site. 

Sat,  Jan  24th 

Remediated  targets  perforated  by  the  remediation  team. 

Each  target  to  be  reinvestigated  was  reacquired  using  first-order  point  Si  A  for  the  GPS  base 
station  as  in  the  original  MTADS  surveys  and  remediation.  The  first  target  in  the  target  list  was 
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point  SIB  to  check  the  reliability  of  our  GPS  set-up.  This  point  was  reacquired  within  5  cm  each 
morning  before  surveying  began.  After  acquisition  of  the  targets,  they  were  marked  with  paint,  a 
wooden  stake,  and  survey  flagging. 

Using  a  web  template,  a  5  x  5-m  area  was  marked  around  each  target  aligned  roughly  N-S.  An 
MTADS  man-portable  survey  was  performed  within  this  area  as  shown  in  Figure  2 1 .  The  man- 
portable  sensors  are  spaced  25-cm  apart  and  survey  lines  were  50-cm  apart.  The  resulting  survey 
data  was  processed  as  described  above  for  the  vehicular  data  and  dig  decisions  made  for  each 
cell  depending  on  the  survey  results.  In  total,  47  cells  were  surveyed  (comprising  the  original 
forty  targets  plus  ten  from  the  extra  lists;  three  of  the  cells  contained  two  targets)  and  35  were 
marked  for  remediation.  The  results  of  these  analyses  are  summarized  in  Table  19  and  the 
details  of  the  analysis  and  remediation  are  given  in  Appendix  F.  Of  the  targets  investigated,  we 
would  categorize  three  as  original  remediation  error.  Target  53,  a  BDU-33  at  4.5  feet,  was 
clearly  missed  in  the  original  remediation.  Targets  56  and  61  were  originally  marked  “no  find” 
but  we  discovered  fence  wire  mixed  with  the  spoil  dirt  of  the  original  hole.  Several  other  targets 
corresponded  to  surface  wire  (particularly  pin  flag  wire).  Give  the  number  of  cattle  on  the  site 
and  the  elapsed  time  since  the  original  survey,  no  conclusion  can  be  drawn  on  these  targets. 


Fig.  21  -  Man-portable  survey  of  the  targets  being 
reinvestigated.  The  survey  cell  is  marked  by  the  5  x  5-m  web 
template. 


Table  19.  Summary  of  January  2004  Remediation  of  Previous  “No  Finds.” 


No  Man-Portable 
Anomaly 

No  Find 

Wire  (comm., 
fence,  or  barbed) 

Hot 

Rocks/Soil 

Ferrous 

Metal 

Number 

12 

17 

8 

9 

6 

4.3.5  Remediated  Targets  at  S2  and  S7 

Fifty  targets  were  remediated  at  each  of  the  two  auxiliary  targets  based  on  earlier  survey  results 
from  Oak  Ridge  National  Lab.  The  dig  lists  are  included  in  Appendix  G. 
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5.  Cost  Assessment 


5.1  Cost  Tracking 


Table  20.  Itemized  Costs  for  the  Combined  Vehicular  and  Airborne  Surveys 


Cost  Category 

Sub-Category 

Cost 

Sub-Total 

Start-Up  Costs 

$89,800 

Preliminary  Site  Visit 

$8,000 

Test  Plan  Preparation 

$15,000 

Equipment  Prep  and  Packing/Unpacking 

$8,000 

Rental  Trailer  and  Transportation 

$13,500 

Analysts  Set-up 

$12,500 

Travel  for  5  Personnel 

$5,000 

Helicopter  Mobilization 

$27,800 

Logistics 

$24,200 

Establish  GPS  Control  Points 

$14,600 

Office,  Garage  Trailers  and  Portable  Toilets 

$2,350 

Generator/Fuel/Electrician 

$4,000 

Materials 

$3,250 

Operating  Costs 

$82,900 

Supervisor 

$12,650 

On-site  Analysts 

$18,000 

Helicopter  Back  Seat  Operator 

$7,700 

Per  Diem 

$4,000 

Rental  Vehicles 

$3,500 

Helicopter  and  Pilot  Expenses 

$20,000 

Jet  A  Fuel 

$2,900 

Field  Labor 

$5,250 

Equipment  Repair 

$8,900 

Analysis  &  Reporting 

$49,500 

Target  Lists,  Dig  Sheets,  Final  Report 

$49,500 

Remediation 

$153,500 

Total  Costs 

$399,900 
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5.2  Cost  Analysis 


The  cost  of  the  combined  vehicular  and  airborne  survey  of  this  area  was  $  175/acre.  Part  of  the 
reason  for  this  relatively  high  per  acre  cost  is  the  fact  that  the  start-up  costs,  which  were  higher 
than  normal  due  to  the  bad  weather  encountered  by  the  aircraft  during  both  halves  of  the 
mobilization,  were  only  amortized  over  1408  acres  ($65/acre).  On  a  larger  area,  the  start-up  and 
reporting  costs  could  be  spread  wider  easily  bringing  the  per  acre  cost  below  $150.  Even  though 
there  was  complete  overlap  between  the  areas  covered  by  the  two  systems  this  is  still  a 
reasonable  guide  for  a  live  site  survey  where  there  will  be  regions,  like  the  eastern  edge  of  this 
site,  where  the  airborne  system  cannot  effectively  survey  and  must  be  covered  by  the  ground 
system.  We  recently  completed  just  such  a  combined  survey  at  the  Badlands  Bombing  Range 
Impact  Area16  where  the  per  acre  cost  was  just  below  $100  because  of  favorable  weather,  site 
conditions,  and  logistics  requirements. 


6.  Implementation  Issues 


6.1  End-User  Issues 

The  primary  end-users  of  this  technology  will  likely  be  site  managers  of  DoD  ranges  and  bases. 
Although  the  equipment  would  be  operated  by  a  remediation  contractor,  no  individual  contractor 
has,  so  far,  been  able  to  make  a  business  case  for  investing  in  this  technology  given  the  uncertain 
DoD  funding  environment.  The  start-up  costs  for  a  commercial  version  of  the  MTADS  airborne 
system  are  approximately  $400K  (over  half  of  which  is  devoted  to  the  sensors  and  sensor 
electronics).  Until  a  reliable  funding  stream  is  in  place,  the  best  option  for  transition  of  the 
technology  is  through  a  process  where  the  NRL  equipment  is  offered  to  a  successful  bidder  as 
GFE  on  a  range  contract.  This  will  allow  the  contractor  to  gain  familiarity  with  operation, 
advantages,  and  disadvantages  of  the  system. 
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